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DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
states. 

2. Claims 1-4, 7-15, 17-24 and 27-33 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Jain et al. (U.S. Patent No. 5, 745, 126). 

3. In regard to claim 1 Jain et al. teaches a method and system in which a viewer 
can command the selection of real, or even the synthesis of virtual, video images of the 
scene in response to any of his or her desired and selected (i) spatial perspective on the 
scene, (ii) static or dynamically moving object appearing in the scene, or (iii) event 
depicted in the scene, (col. 7, lines 37-42; col. 8, lines 41-51). Jain et al. teaches 
synthesizing a virtual video camera, and a virtual video image, from multiple real video 
images obtained by multiple real video cameras (col. 1 , lines 24-27). 

Jain et al. teaches that a computer receives from a prospective user/viewer of the 
scene a user/viewer-specified criterion relative to which criterion the user/viewer wishes 
to view the scene . From a (i) 3D model and (ii) the criterion, the computer produces a 
particular 2D image of the scene that is in accordance with the user/viewer-specified 
criterion. This particular 2D image of the real-world scene is then displayed on a video 
display to the user/viewer (col. 9, lines 8-16). However, the computer is not limited to 
selecting from the 3D model a 2D image that is, or that corresponds to, any of the 
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images of the real-world scene as was imaged by any of the multiple video cameras. 
Instead, the computer may synthesize from the 3D a completely new 2D image that is 
without exact equivalence to any of the images of the real-world scene as have been 
imaged by any of the multiple video cameras (col. 9, lines 57-61). It is noted said 2D 
images is considered to comprise a view volume with a defined field of view. 

Jain et al. illustrates in Fig. 13 a graphical illustration showing the intersection 
formed by the rectangular viewing frustum of each camera scene onto the environment 
volume. The filled frustum representing possible areas where the object can be located 
in the 3D model while, by use of multiple views, the intersection of the frustum from 
each camera will closely approximate the 3D location and form of the obiect in the 
environment model (col. 11, lines 66-67; col. 12, lines 1-7). 

Jain et al. teaches that a viewer interface permits said viewer to select the 
perspective that he or she wants (col. 18. lines 7-8), wherein said interface includes a 
3D cursor that moves in 3D space (col. 22. lines 31-35). It is noted that said cursor Is 
considered to represent a participant (e.g., user/viewer) within said virtual environment . 

Jain et al. teaches that the footprint of a bounding box is proiected to the primary 
surface of motion by intersecting a rav drawn from the optic center of that particular 
camera through the foot of the bounding box with the ground surface. All supporting 
observations are used (with appropriate weighting based on distance from the camera, 
direction of motion, etc.) to update the position of each object (col. 33, lines 53-65). 
Jain et al. implicitly teaches determining a proximity of a representation of an observed 
object in said virtual environment to said view volume. For example, as disclosed 
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above Jain et al, teaches a plurality of viewing frustums specific to respective cameras 
from which said viewing volume is generated. Whether or not said viewing volume is a 
view volume derived from one of said cameras or generated synthetically from a 
plurality of said cameras said volume has a bounds and the obiects contained within 
said bounds must be determined to be rendered within said bounds. It is noted that the 
respective claim language is silent as to the value or range of values that define said 
proximity. 

Jain et al. implicitly teaches processing a view dependent visual data stream of 
said observed object only when said representation is within a specified proximity to 
said view volume. For example, an observed obiect (e.g., a tracked football player - 
col. 7. lines 51-64) located within a given displayed view volume (e.g., view volume of a 
camera - col. 7, lines 65-67: col. 8. lines 1-9) would only be displayed while said obiect 
is contained in said view volume as said view volume is the currently displayed view 
volume. 

4. In regard to claim 2 Jain et al. teaches synthesizing a virtual video camera, and a 
virtual video image, from multiple real video images obtained by multiple (viewpoints) 
real video cameras (col. 1 , lines 24-27). Objects of interest in the scene are identified 
and classified in these 2D images. These multiple 2D images of the scene, and their 
accompanying object information, are then combined in a computer into a 3D video 
database, or model, of the scene (col. 8, lines 55-60). The ultimate response of the MPI 
video system is to synthesize the exact synthetic image, and image sequence, the 
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viewer desires and demands. Even here, no image can be formed where no source 
image data exists (col. 19, lines 12-16). 

5. In regard to claim 3 the rationale disclosed in the rejection of claims 1 and 2 is 
incorporated herein. It is noted that a new view synthesis technique is considered to 
read on a new viewpoint (e.g., the selection of a new viewpoint by said user). 

6. In regard to claim 4 it is noted said display information is sent to said user for 
display (Fig. 1, element 18). 

7. In regard to claim 7 Jain et al. teaches that an object that is out of view, too 
small, and/or occluded from view in one camera is in view, large and/or un-occluded to 
the view of another camera (col. 34, lines 43-45). Jain et al. further teaches that camera 
handoff should be understood to be the event in which a dynamic object passes from 
one camera coverage zone to another. The multi-perspective perception system must 
maintain a consistent representation of an object's identity and behavior during camera 
handoff (col. 29, lines 59-63). It is noted that when said camera handoff occurs that the 
display from the camera from which focus has been passed is ceased to be displayed. 

8. In regard to claim 8 Jain et al. teaches maintaining video information even though 
it is no longer associated with a given displayed object - e.g., hysteresis (col. 8, lines 
55-60; col. 14, lines 60-64; col. 18, lines 27-30). It is implicitly taught that Jain et al. is 
able to determine in advance if video information is to be needed - e.g., anticipation 
(col. 25, lines 5-11). It is noted that the area surrounding, but not necessarily displayed 
(e.g.. the entire football field), for said object (e.g., a tracked football player) is 
considered to read on an extended boundary. 
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9. In regard to claim 9 tlie rationale disclosed in the rejection of claim 3 is 
incorporated herein. It is noted that when a perspective is selected by a user, be said 
selected perspective the first perspective chosen or a different perspective then the 
current one, that a view volume is generated for said respective perspective. 

10. In regard to claim 10 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

11. In regard to claim 1 1 it is noted a tracked football player is considered a local 
participant (e.g., said football player is localized to the respective football game said 
player is participating in). 

12. In regard to claim 12 Jain et al. teaches a virtual collaborative environment that 
be utilized by a plurality (N-way) of users (col. 14, lines 43-59). 

13. In regard to claim 13 the rationale disclosed in the rejection of claim 1 is 
incorporated herein. 

14. In regard to claim 14 the rationale disclosed in the rejection of claim 3 is 
incorporated herein. 

15. In regard to claim 15 the rationale disclosed in the rejection of claim 4 is 
incorporated herein, 

16. In regard to claim 17 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

17. In regard to claim 18 the rationale disclosed in the rejection of claim 8 is 
incorporated herein. 
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18. In regard to claim 19 the rationale disclosed in the rejection of claim 10 is 
incorporated herein. It is noted the resulting display from said change of camera 
(change of location of said user/viewer) is considered to read on a change of view 
volume. 

19. In regard to claim 20 the rationale disclosed in the rejection of claim 10 is 
incorporated herein (e.g., said tracked football player). 

20. In regard to claim 21 Jain et al. teaches that a configuration of the MPI football 
video/television system is shown in Fig. 3. The current system consists of a UNIX 
workstation, a laser disc player, a video capture board, and a TV monitor and graphical 
display (col. 21 , lines 37-42). Jain et al. teaches that said system utilizes 
microprocessors (col. 18, lines 52-56) and that said method is implemented by a 
programmed computer processes (col. 17, lines 51-67; col, 18, lines 1-44). It is 
implicitly taught that said processes, to be executed by a processor(s), are stored on 
computer readable medium (e.g., memory) connected to said processors as said 
processes must be stored in some form computer readable medium as well as able to 
be delivered to said processors to be processed. 

21 . In regard to claim 22 the rationale disclosed in the rejection of claim 2 is 
incorporated herein. 

22. In regard to claim 23 the rationale disclosed in the rejection of claim 3 is 
incorporated herein. 

23. In regard to claim 24 the rationale disclosed in the rejection of claim 4 is 
incorporated herein. 
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24. In regard to claim 27 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

25. In regard to claim 28 the rationale disclosed in the rejection of claim 8 is 
incorporated herein. 

26. In regard to claim 29 the rationale disclosed in the rejection of claim 9 is 
incorporated herein. 

27. In regard to claim 30 the rationale disclosed in the rejection of claim 10 is 
incorporated herein. 

28. In regard to claim 31 the rationale disclosed in the rejection of claim 1 1 is 
incorporated herein. 

29. In regard to claim 32 the rationale disclosed in the rejection of claim 12 is 
incorporated herein. 

30. In regard to claim 33 the rationale disclosed in the rejection of claim 1 is 
incorporated herein. Jain et al. teaches that said method is implemented by a 
programmed computer processes (col. 17, lines 51-67; col. 18, lines 1-44). It is noted 
said processes are considered to read on instructions. It is implicitly taught that said 
processes, to be executed by a processor(s), are stored on computer readable medium 
(e.g., memory) as said processes must be stored in some form computer readable 
medium to be processed. 

Claim Rejections - 35 USC § 103 

31 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

32. Claims 5, 6. 16, 25 and 26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jain et al. (U.S. Patent No. 5, 745, 126), as applied to claims 1-4, 7- 
15, 17-24 and 27-33, in view of Sullivan (U.S. Patent No. 6, 100, 862). 

33. In regard to claim 5 Jain et al. fails to explicitly teach wherein said view volume 
comprises a series of expanding cross-sections of a geometric object. Sullivan teaches 
a multi-planar volumetric display (MVD) system and method of operation are disclosed 
which generate volumetric 3D images (col. 2, lines 56-58). Sullivan teaches wherein a 
view volume comprises a series of expanding cross-sections of a geometric object (col. 
6, lines 22-33; Fig. 1). It is noted that, at least in part, said user-selected perspective is 
considered to determined a view direction. 

It would have been obvious to one skilled in the art, at the time of the Applicant's 
invention, to incorporate the teachings of Sullivan into the system taught by Jain et al., 
because through such incorporation it would provide an improved means of presenting 
the display information generated by Jain et al. as said information would be presented 
in a more realistic, e.g., 3D, manner. 

34. In regard to claim 6 Jain et al. teaches that the system receives and processes 
multiple video views (images) generally of the football field (col. 7, lines 55-57). It is 
noted a football field is considered a four-sided rectangular plane. 

35. In regard to claim 16 the rationale disclosed in the rejection of claim 5 is 
incorporated herein. 
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36. In regard to claim 25 the rationale disclosed in the rejection of claim 5 is 
incorporated herein. 

37. In regard to claim 26 the rationale disclosed in the rejection of claim 6 is 
incorporated herein. 

Response to Arguments 

38. The prior 35 U.S.C. 112 second paragraph rejection has been withdrawn in lieu 
of Applicant's remarks. 

39. In response to Applicant's remark that Jain et al. teaches away from the 
respective limitations disclosed in claim 1 the Examiner does not agree. The Applicant 
is directed to the above rejection of claim 1 which has been highlighted to identify key 
elements considered relevant to said remarks. The Applicant cites a situation taught by 
Jain et al. in which a viewer can ask to be shown a synthesized video view such as from 
a perspective constantly positioned behind a certain offensive running back. It is not 
clear to the Examiner how this is evidence (said cited situation) teaches away from the 
respective claim limitations. If the Applicant takes issue with how a given participant, 
within said virtual environment, is represented (e.g., via a graphic cursor designating a 
given FOV within a virtual environment) it is noted that the respective claim language is 
silent as to any limitations directed toward how said participant is graphically 
represented in said virtual environment. 

40. Applicant's remarks have been fully considered but are not deemed persuasive. 
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Conclusion 

41 . The prior art made of record and not relied upon is considered pertinent to 
Applicant's disclose: Foley et al. (specifically - p. 237-242; Fig. 6.46). 

42. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter-Anthony Pappas whose telephone number is 571- 
272-7646. The examiner can normally be reached on M-F 9:00am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding tlie status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more Infomiation about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (E6C) at 866-217-9197 (toll-free), if you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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